Skip to content

Dbr 3198 updat docker compose - #9

Open
sonam0108 wants to merge 4 commits into
mainfrom
DBR-3198-updat-docker-compose
Open

Dbr 3198 updat docker compose#9
sonam0108 wants to merge 4 commits into
mainfrom
DBR-3198-updat-docker-compose

Conversation

@sonam0108

Copy link
Copy Markdown

#what
the docker compose has old image and old code

#why
just updating the yaml with new code with kafka image and eliminating zookeepers from the code

#how
setup now uses Apache Kafka 3.8.0 with KRaft mode

#testing
I tried to test on my local the connection looks okay

KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:19092
KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
KAFKA_CONTROLLER_QUORUM_VOTERS: 1@localhost:19093,2@localhost:29093,3@localhost:39093

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i dont think this is needed

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

without this setting kafka fails to start with error

"Exception in thread "main" org.apache.kafka.common.config.ConfigException:
If using process.roles, controller.quorum.voters must contain a parseable set of voters.
at kafka.server.KafkaConfig.validateNonEmptyQuorumVotersForKRaft

as we have removed zookeeper and started using Kraft which requires KAFKA_CONTROLLER_QUORUM_VOTERS:1@kafka:9093 referencing ID 1

Comment thread cluster-example/docker-compose.yml Outdated
Comment thread cluster-example/docker-compose.yml Outdated
KAFKA_LOG_RETENTION_BYTES: 26214400
KAFKA_COMPRESSION_TYPE: zstd
KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: "true"
KAFKA_LOG_DIRS: /tmp/kraft-combined-logs

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not needed

Comment thread cluster-example/docker-compose.yml Outdated
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: localhost:22181,localhost:32181,localhost:42181
KAFKA_NODE_ID: 1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i dont think this is needed

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is required for Kraft after eliminating zookeeper we need it for Kraft with a unique ID, as we have KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER if not set its assumed zookeeper mode
https://kafka.apache.org/39/operations/kraft/#zookeeper-to-kraft-migration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants